Forest video monitoring systems intended for detecting and locating forest fires have found their use quite recently. However, they are becoming more and more relevant since a problem with forest fires may arguably be regarded as one of the most serious issues that are currently unresolved by people. Forest fires break out and cause extensive damage in lots of countries all over the world, which may be evidenced by forest fires that broke out within the territory of the Russian Federation in the summer 2010. Those fires had disastrous effects, one of reasons for that being a failure to detect and locate them early, which was repeatedly and explicitly exposed in the media.
The basic structure of the “classical” forest video monitoring system is illustrated below, with reference to FIG. 1. Well-known examples of such forest video monitoring systems include but are not limited to such systems as ForestWatch (Canada), IPNAS (Croatia), FireWatch (Germany). Similar systems such as Klyon (which means “Maple” in Russian) and “Baltika” (which stands for “Baltic” in Russian) have been developed in the Russian Federation.
Forest video monitoring system 100 illustrated in FIG. 1 generally includes a number of remotely controlled video monitoring points 110 and at least one automated operator workstation 120 connected to them for proper operation of video monitoring points 110.
Equipment 120 of automated operator workstation is generally implemented on the basis of widely known computer and communication technologies and typically includes a computer configured to share data remotely and having dedicated and general-purpose software installed in it. General-purpose hardware and software (such as an operating system) that are a part of such a computer are widely recognized in technology. In this regard, the term “computer” may be applied to a personal computer, a laptop, a set of interconnected computers with features that meet requirements established for computer system 100. A computer is connected to a display device which shows a graphical user interface (GUI) associated with a specialized application. The GUI enables operators to perform their work related to visual surveillance of the territory and management of video monitoring points 110. Interaction with elements of the graphical user interface is carried out with the help of well-known input devices connected to a computer, such as a keyboard, a mouse, etc.
Every video monitoring point 110 is actually a piece of transmission equipment 111 placed on a tall structure 112. Tall structure 112, as a rule, may be any tall structure that meets requirements established for system 100 (i.e. a structure that is fit for placing transmission equipment at a sufficient height and that provides an opportunity to observe quite a vast territory) and is usually a tower put up by a communication provider or a mobile carrier, or a television tower, or a light tower, etc.
A generic term “transmission equipment” 111 is used in relation to equipment placed on tall structure 112. Such equipment includes a controlled video unit 113 and a communication module 114 for communication/data exchange with operator workstation(s) 120.
Controlled video unit 113 normally comprises a digital video camera 115 equipped with a zoom system 116 and mounted on a rotating device 117 which can be used to mechanically change spatial orientation of camera 115 with high precision.
Transmission equipment 111 also includes a video camera control equipment 118 connected to communication module 114, video camera 115, zoom system 116, and rotating device 117 and designed for general control of functions performed by the controlled video unit 113 as a whole and by its components in particular. Thus, upon receipt of control signals from the operator through communication module 114, control unit 118 is designed to set required spatial orientation of video camera 115 (e.g. in order to point it at an object that must be monitored) by controlling rotating device 117 and/or zoom in/out on an image of the object monitored with it by controlling zoom system 116. In addition, control unit 118 is designed to determine the current spatial orientation of camera 115 and provide data on its current spatial orientation through communication module 114 to a requesting party (in particular, to operator workstation 120, where these data shall be displayed in the GUI, for instance). Functionalities listed here are known properties of modern sets of controlled video cameras offered on the market.
Control unit 118 is generally a hardware unit based on microprocessors such as a controller, microcomputer, etc. and obvious to a specialist. It is programmed and/or programmable in a certain manner to perform functions assigned to it. Control unit 118 may be programmed, for instance, by recording (embedding) its firmware, which is widely known in technology. Accordingly, video camera control unit 118 is normally connected to a memory unit (such as an integrated flash memory) which stores corresponding (micro) software that works to fulfill functions associated with control unit 118.
Workstations 120 may be connected to video monitoring points 110 both directly and via a communication network (such as network 130) using widely known and applied wired and/or wireless, digital and/or analog communication technology, while communication module 114 of video monitoring point 110 and computer communication interface of operator workstation 120 must meet communication standards/protocols which such connection is built on.
Thus, an illustrative network 130 which is connected to video monitoring points and automated operator workstations 120 may be an address network, such as the Internet. If there is a data channel belonging to a third-party provider available on the site where video monitoring point 110 is installed, which is frequently encountered, it is preferable to use this channel to connect transmission equipment 111 to the Internet. If it is not possible to connect video monitoring point 110 to the Internet directly on the site where it is installed, widely known wireless broadband technology (such as Wi-Fi, WiMAX, 3G, etc.) shall be used to provide communication between transmission equipment 111 and an Internet access point. Operator workstations 120 are connected to network 130 in a similar way. In particular, depending on the access technology applied, a modem (including a wireless one), a network interface card (NIC), a wireless access card, etc., being whether external or internal in relation to operator computer workstation 120, may be used in order to connect to network 130.
As a rule, system 100 also includes server 140 connected to network 130. Functions of centralized control over a set of video monitoring points 110 and over their interaction with operator workstations 120 are delegated to this server in order to ensure reliable operation of system 100. Server 140 is typically a high-performance computer or a set of interconnected computers (such as a rack of blade servers) with specialized server software installed on it (them) which has (have) high-speed (e.g. optical) connection to the Internet. Hardware/software implementation of such a server is obvious to an expert. In addition to general functions related to controlling system 100, server 140 can perform various highly specialized functions—for example, it can function as a video server which ensures collection and intermediate processing of data and provision of such data upon user's request.
With a forest video monitoring system arranged in this manner, one user can monitor the territory controlled, while managing multiple cameras at the same time. Besides, due to the above-described typical functionalities it is possible to automatically locate a fire source rapidly when monitoring with multiple video cameras by using a widely known angle-measuring method and store predetermined patrol routes in a memory (for instance, on server 140 or in a computer of operator's workstation 120) in order to access them promptly and carry out monitoring procedures. In this context, a “patrol route” implies a predetermined sequence of camera orientation changes meant for providing visual information related to a predetermined territory required.
It should be noted that capacity of modern electronic hardware enables you to use it as a basis for creating devices meant for visualization and control out of components that are part of the forest video monitoring system having fairly broad user functionality, which greatly simplifies operator's work. In addition, modern hardware, with the aid of software specially executed by it, can take over some functions related to automatic detection of potentially hazardous objects on videos or still images obtained from video cameras (in terms of forest monitoring, such objects may include but are not limited to smoke, fire, etc.). These computer vision systems designed for searching an image for hazardous objects can use prior information about features of smoke or fire such as specific movement, color, brightness, etc. Such computer vision systems are applied in a number of industries, ranging from robotics to security systems, which is quite thoroughly described in, e.g., publication Computer vision. A modern approach written by D. Forsyth and J. Ponce, Williams Publishing House, 2004, 928 pages.
Such an intelligent subsystem that implements the specified computer vision technology in general can also be implemented both at operator workstation 120 and at server 140, and even in controlled video unit 113.
The abovementioned information is a summarized structural description of the classical system for video monitoring of forests whose principle of operation is based on the use of controlled cameras. Its additional aspects directly related to identifying and processing coordinates of objects detected are reflected in more detail, in particular, in patent publications RU 2458407, WO 2012/118403.
It should be noted that creation and deployment of such forest video monitoring systems has become possible only in recent years. Only present days have seen the number of cell phone towers sufficient for covering main fire hazardous spots. In addition, broadband Internet access which enables us to share large volumes of information and transmit real-time videos via the Internet has become far more available and the cost of equipment ensuring wireless communication over long distances has been reduced.
However, with reference to the most urgent goal of monitoring vast areas (i.e. areas the size of a federal subject or the size of the country) when constructing a forest video monitoring system, the following significant problems should be considered.
A large number of PTZ-cameras required as a part of the system makes it difficult to monitor any area: for example, an operator may miss a section observed by a camera and it may take a while for the next inspection of the site to occur. Therefore, automation systems and automatic detection systems should be used, with such automation of the detection procedure requiring considerable computational resources and transmission of large amounts of data.
However, mere expansion of computational capacities within this framework is not a cure-all solution as there appears an exceptionally challenging task to ensure appropriate communication infrastructure for transmitting data from video monitoring points (such as video monitoring points 110 in FIG. 1) to computers at operator workstations (120) and/or the server (140) due to the fact that considerable spatial distribution and strong heterogeneity is, in fact, inherently typical of such required communication infrastructure.
So, for the time being, technical feasibility to connect video monitoring points to data networks is, as a rule, quite limited. In particular, in order to have a Full HD standard video stream broadcast from a video camera, it is implied by default that there is a communication channel available having a bandwidth of about 4 Mbit/s, however, in some points on the way from the camera to the operator only communication channels with a bandwidth of about 100 kbit/s are provided, which means by default that it is impossible to arrange direct video surveillance.
When looking at the problems outlined above, which arise when solving a problem of monitoring vast territories, it becomes evident that classical systems for forest video monitoring described above are poorly fit for that.
For instance, due to the fact that classical systems for forest video monitoring are “locally” focused, principles of their construction do not imply sufficient flexibility and suggest that there is only one server available where all data processing procedures are carried out. This approach, in its turn, suggests that there must be such a communication infrastructure whose features shall be sufficient for transporting data from cameras, analyzing them, and delivering analysis results to end consumers. As mentioned above, experience of building systems for monitoring vast areas shows that any communication infrastructure, considerably spread geographically, always has bottlenecks (i.e. areas with insufficient throughput capacity) which reduce general efficiency of data delivery. That fact alone makes the use of classical schemes of building video monitoring systems both technically and economically unfeasible for vast areas due to the fact that in order to eliminate such bottlenecks it is required to arrange new high-speed communication channels on a large scale, including deployment of new equipment, etc. and/or costly rental of existing high-speed communication channels provided by third parties.
In addition, with implicitly intense automated processing of video data, it is also problematic to concentrate such processing procedures on one server due to limitations of computational resources of a single server.
In order to illustrate the foregoing, FIG. 2 shows an approximate need for various components of classical system 100 for forest video monitoring according to FIG. 1 in terms of communication channels where the arrow width nominally meets a required throughput capacity of an input/output communication channel (not to scale). In this case, the term “Internet” implies a digital data transmission network which has virtually no speed limits.
Thus, technology is in need of building a distributed architecture for a forest video monitoring system within a large area which would ensure efficient arrangement of data traffic, eliminating the need for widespread use of expensive high-speed channels within any controlled large area, and provide flexible management of computational resources required for analyzing the data obtained.